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FOREWORD 


The  purpose  of  thfs  report  is  to  present  selected  findings 
resulting  from  research  on  contract  cost  growth  and  to  indicate 
the  future  directions  to  be  taken  in  studying  cost  growth.  It  is 
assumed  that  the  reader  is  familiar  with  the  March  1971  Amy  Procure¬ 
ment  Research  Office  publication,  “Production  Cost  Growth"  and  has 
a  basic  understanding  of  procurement  and  the  subject  of  cost  growth 
in  general.  This  report  contains  results  of  further  analyses  of  the 
data  obtained  in  the  DD  Form  1500  and  Contractor  Performance  Evalu¬ 
ation  Forms. 
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ABSTRACT 


This  Is  h  follow-on  report  to  the  Production  Cost  Growth  Study 

dated  March  1971.  Results  of  the  analysis  of  cost  growth  of  major 

Army  weapon  procurements  relative  to  the  contract  size,  duration, 

technology  level,  and  Inflation  are  presented  in  some  detail. 

The  results  of  this  study  indicate  that  the  contract  duration 

has  a  highly  significant  effect  on  cost  growth  vdrile  the  contract 

size  has  a  very  mild  effect.  Inflation  by  itself  was  found  to  have 

* 

a  small  effect  on  cost  growth.  It  was  found  that  the  contracts  which 
involve  the  highest  level  of  technical  sophistication  experience 
significantly  more  cost  growth  than  the  others  for  Army  contracts; 
but  this  was  not  found  for  the  Navy  or  Air  Force  contracts. 

This  report  also  describes  the  data  collected  for  the  next 
segment  of  research  on  this  cost  growth  project  and  previews  the 
next  scheduled  report. 
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SUGARY 


1.  Rrobles . 

Contract  cost  Increases  in  ODD  have  cose  under  particularly  close 
scrutiny  within  the  last  decade,  resulting  in  criticism  free  many 
quarters.  The  implication  of  this  criticise  seesas  to  be  inefficiency 
at  best,  or  basic  dishonesty  at  worst.  ODD  is  naturally  sensitive  to 
this  type  of  criticism  and  has  reacted  strongly.  Many  studies  and 
seminars  have  been  spensored  for  the  analysis  of  the  procurement 
process.  Partly  as  a  result  of  this,  changes  in  the  unwieldy  pro¬ 
curement  machinery  have  been  implemented  which  add  a  tremendous 
amount  of  complication  to  the  existing  process.  Cost  growth  in 
programs,  systess  and  the  individual  component  contracts  still  re¬ 
main.  It  setsss  that  the  basic  underlying  causes  of  cost  growth  are 
mt  cessp! ®fc@1y  known.  If  these  ceases  are  in  fact  known,  then  we  in 
900  are  ssrely  covering  our  tasks  by  sponsoring  studies  aimed  at 
discovering  causes.  It  seems  snore  likely  that  most  of  the  grass- 
root  causes  are  known  -  such  as  Inefficient  cost  estimating  procedures, 
unpredictable  technical  proble&s  and  the  like  -  but  the  relationship 
among  tht  aany  causes  is  so  complicated  that  the  machinery  cannot  be 
controlled  at  present. 

2.  Background  „ 

The  Arso'  Proeursassnt  Research  Office  has  been  assigned  the  study 
of  cost  growth  in  the  procurement  of  sajor  Amy  weapon  systems.  The 
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overall  study  Is  statistical  la  nature,  using  the  data  contained  in 
the  I®  1500  For?-  -‘or  all  contracts  (excluding  FFP)  over  $1  ,000,000, 
Contractrr  ?erfomanee  Evaluation  Reports,  and  selected  contract  file 
data.  The  results  of  the  first  phase  of  the  study  are  contained  In 
the  "Production  Cost  Growth*4  Report  of  March  1071.  That  research 
.volved  around  par&rjeters  endogenous  to  the  procurement  (contract  type, 

•  cnodltf,  and  suggested  several  areas  for  continued  study. 

3bj  , 

The  obj  ;  of  this  phase  of  the  research  is  to  determine  whether 
the  following  factors  contribute  significantly  to  cost  growth:  contract 
durat-  s  (from  time  of  definitlzation  to  time  of  close  out),  definltlzed 
contract  dol’ir  amount.  Inflation,  and  the  level  of  technology  involved 
In  the  work. 

4.  Data  Base. 

\  .>2  data  described  in  this  report  is  found  In  the  00  1500  Forms 
for  all  contracts  over  $1,000,000  between  1958  and  1969,  and  in  the 
data  In  AMC  CPE  files  for  the  period  1966-1970.  The  Forms  1500  Jsta 
were  used  for  the  analysis  of  cost  growth  with  respect  to  contract 
initial  cost,  contract  duration  and  Inflation,  There  were  580  FPI, 

CPIF,  and  CPFF  contracts  in  this  data  set.  The  CPE  data  Is  composed 
of  58  Amy,  30  iiavy,  and  113  Air  Force  contracts,  and  contains  a 
much  finer  breakdown  of  the  contract  cost  information.  Therefore 
this  data  set  was  necessary  for  that  part  of  the  study  relating  to 
the  technology  level. 
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5*  Conclusions. 


The  foldings  Indicate  that  the  contract  duration  and  Initial 
cost  both  bsv-  an  effect  on  cost  growth,  but  were  found  not  to  be 
related  to  each  other.  Osher  factors  such  „  ,nflatton  Jm| 

®f  modifications  Involved  *.  the  contract,  are  strongly  related  to 
duration  so  that  all  of  te.se  effects  must  be  studied  teparately  to 
assess  the  individual  contribution  of  each  to  cost  growth,  m  fact. 
Inflation  was  found  w  bare  a  very  small  but  statistically  significant 
effect.  It  was  found  that,  for  Army  contracts,  there  ere  differences 
between  different  levels  of  technical  difficulty  with  respect  to 
cost  growth  characteristics,  the  highest  level  of  technical  sophisti¬ 
cation  yielding  the  greatest  amount  of  cost  gsowth.  This  was  not 
found  to  be  true  for  either  the  Air  Force  or  the  Navy  contracts. 


CHAPTER  I 


INTRODUCTION 

The  total  amount  of  defense  contract  cost  Increases  of  actual 
money  spent  over  and  above  the  Initial  contract  price,  has  amounted 
to  many  billions  of  dollars  in  the  past  two  decades.  Estimates  of 
the  average  percent  increase  range  as  high  as  400 %  for  R&D  contracts. 
In  one  particular  case,  an  individual  contract  exhibited  a  cost 
Increase  of  11 ,667* J 

There  are  obviously  many  causes  of  this  cost  increase  {not  all 
undesirable),  and  these  have  been  found  to  be  basically  of  5  types 
(listed  here  in  order  of  desirability):  (1)  changes  in  quantity  or 
technical  requirements  after  deflnitization  of  contract;  (2)  Inclusion 
of  new  techniques  which  are  not  known  at  time  of  definitlzation; 

{3)  general  price  Increases  (inflation);  (4)  management  Inefficiencies 
such  as  the  use  of  poor  estimating  procedures  or  lack  of  the  required 
levels  of  technical  ability;  and  (5)  the  buy-in  (accompanied  by  subse¬ 
quent  increases). 

Requirement  changes,  both  in  performance  and  quantity,  cannot 
be  considered  undesirable,  and  changes  in  scope  of  work  which  are 

^Note:  In  this  report  the  terse  cost  growth,  overruns  and 
underruns  are  as  defined  in  Secretary  Packard's  Memo  of  August  5, 

1970  for  the  Secretaries  of  the  Military  Departments. 


1 


designed  to  take  advantage  of  recent  advances  in  technology,  may  in 
fact,  be  highly  desirable.  It  Is  noted  that  quantity  changes  are 
usually  anticipated  as  is  evidenced  by  the  frequent  use  of  contract 
options.  Technology  changes  are  not  always  more  costly  in  them¬ 
selves,  but  contractors  normally  obtain  additional  funds  for  Implementing 
these  changes. 

The  tax  paying  public  is  not  usually  made  aware  of  the  prevalence 
and  importance  of  cost  increases  of  this  kind.  From  the  sensationalism 
of  the  news  media,  the  general  public  is  led  to  believe  that  cost 
increases  are  a  result  of  inefficiencies,  inflation  or  graft.  It  must 
be  admitted  that  the  massive  procurement  machine  is  not  immune  to 
many  of  these  charges.  Unfortunately,  among  the  categories  of  causes 
given  above,  the  "buy-in"  seems  to  be  Important. 

The  policy  of  "buying-in"  on  a  contract  refers  to  a  contractor's 
deliberately  underbidding  all  competition  so  that  he  will  be  assured 
of  obtaining  the  contract.  This  usually  means  that  his  bid  must  be 
lower  than  his  estimated  costs  of  satisfying  the  work  statement.  One 
way  to  recoup  the  losses  incurred  in  this  situation  is  to  obtain 
additional  funds  through  contract  modlfl cations.  For  example,  the 
contractor  may  propose  a  change  designed  to  Increase  performance  at 
a  drastically  Inflated  cost. 

It  would  be  very  optimistic  to  feel  that  all  cost  growth  resulting 
from  inefficiencies ,  buy-ins  and  the  like  could  be  halted  by  pressure 
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from  within  the  government.  As  a  matter  of  fact,  as  an  investi¬ 
gation  proceeds  beyond  the  descriptive  or  conjecture  stage,  the 
difficulty  of  identifying  the  causes  by  relating  then  to  any  con¬ 
crete  piece  of  datum,  bscssses  apparent.  Consequently,  one  is 
forced  to  develop  abstract  or  indirect  methods  of  investigation  for 
such  areas  as  the  effects  of  inflation  or  the  "buy-in".  Some  cost 
growth  aspects  are  particularly  eHclusive.  For  example,  how  does 
one  determine  whether  it  is  the  government  or  the  contractor  who 
actually  initiates  a  particular  contract  modification?  While  the 
change  order  may  indicate  that  it  is  the  government,  other  evidence 
may  cast  doubt  on  this. 

In  the  first  report  (Production  Cost  Growth,  March  1971),  many 
Inherent  factors  related  to  contract  descriptive  characteristics 
(e.g.,  contract  type,  commodity,  type  of  work,  etc,)  were  investigated. 
It  was  found  that  contract  modifications  account  for  93%  of  cost 
growth,  while  overruns  account  for  only  7%.  Furthermore,  cost  growth, 
contract  modifications  and  cost  overruns  all  differ  significantly 
for  RfiD  and  production  contracts.  A  more  detailed  analysis  showed: 

(1)  a  significant  difference  for  total  cost  growth  and  modifications 
on  production  contracts  between  some  contract  types,  and  between 
commodity  classes;  and  (2)  a  significant  d;f*e, -*enre  for  cost  overruns 
on  production  contracts  between  commodity  glasses,  and  between  a  sample 
of  10  individual  contractors. 
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In  this  study  the  effects  investigated  are  more  external  In 
nature.  It  seems  likely  that  Inflation  would  be  a  (minor)  eontrl- 
butor  to  cost  growth.  Furthermore,  cost  growth  sight  be  expect®! 
to  vary  with  the  actual  duration  of  the  contract.  These  effects  as 
well  as  the  contract  size  (dollars)  and  the  level  of  sophistication 
of  the  work  Involved  will  all  be  Investigated.  Part  I  of  the  cost 
growth  study  was  concerned  with  those  effects  which  are  somewhat 
Inherent  to  the  contract  itself.  This  report  Is  sore  closely  aligned 
with  the  study  of  effects  which  are  sere  related  to  contractor  be¬ 
havior  and  the  government  reaction  to  It.  In  other  words,  this  report 
Is  addressing  issues  more  closely  related  to  the  entrepreneurial 
behavior  of  the  contractor. 

Previously,  it  was  mentioned  that  contract  modifications  account 
for  approximately  93 %  of  cost  growth  as  opposed  to  the  7%  due  to 
overruns.  The  most  obvious  question  which  this  raises  is:  What 
types  of  contract  modifications  are  the  main  contributors  to  cost 
growth  and  of  these,  which  can  be  controlled? 

The  data  which  Is  presently  available  in  the  DD  1500  Forms  does 
not  contain  a  listing  of  modifications.  This  Information  is,  however, 
contained  In  the  CPE  (Contractor  Performance  Evaluation)  reports, 
initiated  In  1966.  These  CPE  forms  were  discontinued  In  FY  1971  and 
so  only  a  very  limited  number  of  records  are  available.  Furthermore, 
numerous  inaccuracies  have  been  noted  in  both  the  CPE  and  the  1500 
forms.  This  Imposes  the  requlresent  of  locating  e  new  data  source. 
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Because  of  the  requirement  for  date.  It  teas  decided  to 
schedule  approximately  12  man  weeks  of  travel  devoted  to  the  acquisi¬ 
tion  of  information  from  actual  completed  contract  files  located  In 
the  various  cetmands.  This  Infomatlon  breakdown  is  described  in 
Figure  1  and  Includes  a  complete  breakdown  by  modification  type  and 
amount  for  CPIF,  CPFF,  FPI  and  FFP  type  contracts.  The  FFP  contract 
Information  was  not  previously  available  for  the  study,  and  represents 
new  infomatlon.  The  data  collection  Imposes  the  further 
requi reraent  of  about  60  man  weeks  of  effort  to  transcribe  the  data 
to  coding  forms,  coding  sheets,  IBM  cards  and  finally  to  computer 
tapes. 

In  summary,  the  first  report  presented  tests  of  relationships 
between  contract  characteristics  and  cost  growth;  this  report  presents 
the  results  of  tests  made  on  possible  relationships  between  cost  growth  and 
selected  economic  and  technical  factors;  and  the  next  report  will  present 
results  of  tests  of  relationships  between  cost  growth  and  various  types 
of  contract  modifications. 
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COST  GROWTH 

1.  Contract  Number: _ _ 

2.  Contract  Type:  FF? _  FPI _ _  CPlr _  CPFF 

3.  Contractor: 


4.  Item: _ _ 

5.  Independent  Govenwent  Estimate :  _ . _ _ 

6.  Co^titlve _  Negotiated _  Sole  Source _ 

Cost  Analysis _ 

7.  Initial  Contractor  Proposal  (DD-633): _ 

8.  Initial  Government  Counter  Offer: 

—  <— ■  -  .W  ■  "HI 

9.  Date,  Letter  Contract:  _ _ 

10.  &aount.  Letter  Contract:  _  _ 

11.  Date,  Definltizatlon:  _ _ 

12.  Amount,  Definltlzation: _ 

13.  Incentive  Arrangements : _ 

14.  Type  Procurement: 

Production _  RSD _  Services _ Other 

15.  Negotiation  Authority: _ _ 

16.  Modifications  (list  on  separate  sheets). 


FIGURE  1 
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mmm  m  size  of  contracts 


As  am  to  list  possible  significant  contributors  to 

cost  growth,  &m  of  the  first  that  cases  to  mind  Is  the  tiiee 
i lupafclon  of  t to  contract,  This  is  reinforced  by  the  fact  that 
several  other  apparently  contributing  effects  are  related  to  It. 

F@r  i'K^I»let  Inflation  obviously  has  a  longer  time  span  In  which 
to  take  effect  In  longer  contracts.  Also,  there  Is  acre  time  to 
Introduce  contract  modifications,  the  major  ccntrlbytor  to  cost 
growth.  As  a  matter  of  fact,  duration  would  appear  to  be  related 
to  the  total  dollar  amount  of  the  contract  which  presumably  Is 
related  to  cost  growth . 

Before  the  data  can  be  subjected  to  a  regression  analysis,  it 
should  be  noted  that  while  the  contract  size  in  our  sample  varies 
froa  ST ,000,000  te  about  $55,000,000  and  the  contract  duration  ranges 
fro*  0  to  19  years,  the  percent  cost  growth  varies  from  about  -10 
to  2,000.  This  would  sake  the  slope  of  the  respective  regression 
lines  approximately  . CK>003  and  .0000004,  and  would  almost  certainly 
yield  non-significant  statistics.  In  order  to  overcome  this  difficulty, 
the  contract  sizes  and  durations  are  scaled  downward  by  dividing  by 
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the  average  contract  size  and  duration  for  the  entln*  sample*  and 
the  percent  cost  growth  Is  divided  by  100  yielding  fractional  cost 
growth.  Tn  this  way,  each  scaled  variable  ha<  a  range  of  about  5 
or  10.  This  procedure  will  transform,  the  dat*  such  a  way  that 
any  relationship  between  cost  growth  and  contract  duration  will  be 
detected  for  prescribed  confidence  levels;  however,  if  no  significant 
relationship  exists,  then  none  will  be  Introduced  In  this  way. 

The  details  of  the  proceeding  paragraph  can  be  expressed 
symbolically  in  the  following  way.  Denote  the  Initial,  adjusted  and 
final  contract  costs  by  Cj,  Ca  and  Cf.  Then  the  fractional  cost 
growth,  modifications  and  overruns  are: 

cf~c1  cfca  and  ca’c1 

Cl  Ci  ir¬ 

respectively. 

Let  y-j  represent  the  contract  duration  for  contract  number  1. 

.  i  n 

Then  the  sample  average  duration  Is  y  »  ^  I  y*  and  the  normalized 
contract  duration  for  the  contract  Is: 

y\  s  yi/y  • 

The  results  of  a  regression  analysis  and  the  associated  one  way 
analysis  of  variance  are  given  In  Table  II. 1.  The  contract  Inltla' 
cost  scaled  with  the  average  initial  cost  Is  Included  in  the  regression 
analysis.  The  scaling  yields  cost  data  with  the  approximate  range 
0  to  5. 
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Independent  Variable:  Cost  Growth 


Independent  Variable:  Modifications 
Constant  -2. OS 

Initial  tost  -.166  :097 

Duration  2.065  .363 


Independent  Variable:  Initial  Cost 
Constant  1.054 

Duration  -.054  .155 

F 


Variable 

Coefficient 

Standard  Error 

F  Values 

Constant 

-.265 

Initial  Cost 

-.176 

.093 

3.24* 

Duration 

2.237 

.363 

37.84*** 

Independent  Variable:  Overruns 

Constant 

-.056 

Initial  Cost 

-.010 

.009 

1.29 

Duration 

.172 

.003 

26.54*** 

2.89* 

32.38*** 


.122 


.1  ,  1  ,577  -  2.72 


F.0S,  1,577  *  3.85  F.01 ,  1 ,577  *  6.66 


NOTE:  In  all  of  the  tables  in  this  text  one,  two  or  three  asterisks 
denote  statistics  which  are  significant  at  the  10*.  5*.  or  1% 
levels,  respectively. 


TABLE  n.i 
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The  multi -linear  model  is  used  in  the  regression  analysis; 


\ 


«  a0  ♦  a-jXj  j  +  a^  j  ,  where 

■  fractional  cost  growth,  or  fractional  cost  overruns, 
or  fractional  contract  modlfi cations. 

x-j  *  normalized  contract  duration 

xj>  «  normalized  contract  initial  cost 

*i  *  the  associated  error  (randomness) 

a0  *  the  constant  tem  or  overall  average 

This  regression  analysis  was  backed  up  by  an  analysis  of  variance, 
ANOVA,  in  which  the  contracts  were  divided  into  treatment  groups  by 
duration,  a  total  of  14  groups.  The  results  of  the  analysts  of  variance 
are  given  in  Table  II. 2. 

The  analysis  indicates  that  the  contract  duration  is  strongly 
related  to  cost  growth,  overruns  and  modifications.  While  the  con¬ 
tract  Initial  cost  does  have  a  statistically  significant  effect  on 
modifications  and  cost  growth,  its  effect  is  extremely  mild,  ft 
must  be  remembered  that  the  effect  of  inflation  Is  related  to  duration, 
so  that  this  effect  must  be  Investigated  further  for  significance  as 
one  of  the  underlying  causes.  (See  Chapter  III)  Furthermore,  the 
total  ni-mber  of  contract  modifications  could  reasonably  be  expected 
to  Increase  with  the  time  duration  of  the  contract.  This  will  also 
be  investigated  at  a  later  time  in  this  study. 
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ANOVA:  Cost  Growth  Vs. 

Duration 

SS 

DF 

F 

Between 

2179.9 

13 

4.63*** 

Within 

20486.8 

566 

Total 

22666.6 

579 

Overruns  Vs.  Duration 

Between 

34.7 

13 

9.92*** 

Within 

152.3 

566 

Total 

.87.0 

579 

Modifications  Vs. 

Xiration 

Between 

2031.7 

13 

4.35*** 

Within 

20320.3 

566 

Total 

22352.0 

F.01t  13,500 

«  2.17 

NCTEj  SS  means  sum  of  squares  and  OF  means  degrees  of  freedom 


TABLE  1 1. 2 


It  is  very  surprising  that  the  contract  sl2e  and  duration 
are  not  related  (Table  II, 1),  since  cost  growth  is  related  to  both. 
Intuitively,  it  would  see*  that  larger  dollar  amounts  would  be 
spent  over  longer  time  spans  on  the  average,  but,  this  does  not 
see*  to  be  the  case. 

The  reason  for  this  can  be  seen  in  the  second  column  of  Table 
II. 1.  The  coefficients  for  initial  contract  cost  versus  percent 
cost  growth,  percent  overruns,  and  percent  modifications  are  all 
negative  while  the  corresponding  coefficients  relative  to  the  dura¬ 
tion  are  positive.  This  means  that  the  three  effects  listed  above 
increase  with  contract  duration  (on  the  average)  and  decrease  with 
contract  initial  cost.  This  helps  account  for  the  lack  of  dependence 
between  duration  and  initial  cost,  A  given  change,  say  increase, 
in  the  average  cost  growth,  would  almost  have  to  be  a  result  of  an 
increase  In  duration,  but  this  could  be  accompanied  by  either  a 
large  increase  or  large  decrease  In  Initial  cost. 

While  the  results  of  the  preceedlng  analysis  should  not  be  used 
for  prediction,  owing  to  the  present  indefiniteness  in  the  model, 
some  statements  can  be  made  from  it.  The  regression  equation  from 
Table  II. 1  Is: 

Y*  *  -.265  -  .176  •  +  2.23 7*^-  , 

J  Cf  u 

where  y  *  fractional  cost  growth.  The  appropriate  transformation 
then  yields: 
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%  cost  growth  *  -.265  -.000027  *  Ci  4-  60.5  •  D. 


Tfsis  means  that  for  each  year  increase  In  contract  duration,  there 
is  a  corresponding  60. 52  increase  in  the  rate  of  cost  growth.  Also, 
for  each  million  dollar  increase,  there  is  a  corresponding  decrease 
of  37*  in  the  rate  of  cost  growth. 


CHAPTER  III 
INFLATION 

The  investigation  of  the  effects  of  Inflation  on  cost  growth 
requires  special  attention  because  of  the  lack  of  a  direct  indi¬ 
cator  of  inflation  in  the  contract  file  data.  Usually,  inflation 
is  taken  Into  account  in  major  procurement  pricing  procedures. 
Therefore,  If  the  rate  of  Inflation  does  not  differ  from  that 
anticipated  at  the  time  of  contract  definftlzation,  there  will  be 
no  cost  overrun  or  underrun  as  a  direct  result  of  inflation.  It 
is  reasonable,  then,  to  search  for  a  possible  relationship  between 
overruns  and  the  difference  between  actual  and  anticipated  inflation. 

It  is  impossible  to  tell  from  the  1500  and  CPE  data,  how  much 
inflation  was  anticipated  in  a  given  procurement.  Furthermore,  it 
is  unreasonable  to  assume  that  a  relationship  exists  between  the 
total  dollar  amount  of  contract  modifications  (for  changes  in  scope, 
quantity  Increases,  etc.)  and  inflation.  Tnis  suggests  attempting 
to  obtain  an  indirect  measurement  of  inflationary  effects  (with  an 
apparent  increase  in  the  significance  level  of  the  subsequent  statis¬ 
tical  tests). 

The  main  hypothesis  in  this  section.  Is  that  overruns  are 
positively  correlated  with  increasing  inflation.  It  also  seems 
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reasonable  to  assume  that  estimates  of  "'future"  Inflation,  are 
based  on  past  and  present  inflation  indices.  Finally,  it  is  con¬ 
venient  to  assume  that  if  there  are  other  contributors  to  cost 
overruns,  (such  as  defective  cost  estimates)  then  all  effects  are 
additive  with  respect  to  inflation. 

Obviously,  in  a  quantitative  analysis  of  an  effect  as  compli¬ 
cated  as  inflation  in  the  procurement  process,  it  is  necessary  to 
deyelop  a  model  which  is  easy  to  program  (both  in  time  and  level  of 
sophistication)  and  which  on  the  average,  yields  accurate  results. 
For  the  remainder  of  the  mathematical  analysis  and  the  associated 
assumptions,  the  reader  is  referred  to  Appendix  I. 

The  ANOVA  based  on  the  previous  assumptions,  and  those  in 
Appendix  I,  is  given  in  Table  III .1  below. 


Overrun  Vs.  Inflation 


J  Source 

d.f . 

M.S, 

F. 

R. 

t 

[Treatments 

1 

2.67 

5.85 

.17 

2.42 

Error 

198 

0.46 

Constant  term  =  -.82  t  qi  =  2.345 

Linear  coefficient  *  .82  F  05  =  3.89 


TABLE  III.l 
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The  results  e  encouraging.  The  estimated  correlation 
coefficient  R  *  .,7  is  somewhat  low*  but  this  Is  not  surprising 
because  of  the  gross  assumptions  involved.  Furthermore,  the 
associated  t-value  is  highly  significant  Indicating  that  the 
true  correlation  Is  not  zero  with  99£  confidence. 

There  Is  another  Indication  that  the  regults  are  useful  Notice 
that  the  sum  of  the  constant  term  and  the  linear  coefficient  Is 
zero,  which  should  be  the  case  if  the  model  is  an  accurate  description 
of  Inflation.  In  a  period  of  no  Inflation,  the  Inflation  Index,  I, 
is  equal  to  l.G  {see  Appendix  I).  Since  Inflation  Is  the  independent 
variable  and  the  associated  overrun  is  the  dependent  one,  the  model 
is  y  «  a0  +  a-jX,  where  X  *  the  inflation  index  and  y  »  contract  over¬ 
run.  When  X  =»  1  and  ato  *  -a then  y  *  0.  Tills  means  that  during 
periods  of  no  inflation,  the  expected  overrun  is  zero. 

The  inflation  analysis  was  broken  down  further  by  type  of 
contract  for  the  following  reason.  Inherent  in  the  analysis  is  the 
assumption  that  the  contractor  attempts  to  plan  for  inflation.  The 
different  types  of  contracts  reflect,  to  a  certain  degree,  different 
levels  of  risk  involved  in  the  contract  costs  due  to  uncertainties.  It 
seems  reasonable  to  suppose  that  as  the  risk  increases,  cost  planning 
becomes  more  detailed.  Because  of  the  nature  of  the  contract  mechanism, 
the  following  order,  FFP,  FPI,  CPIF,  and  CPFF  reflects  a  decrease  in 
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risk  to  the  contractor  from  FFP  to  CPFF.  Of  course,  FFP  contracts 
do  not  haye  ths  overrun  option.  In  any  event,  different  overrun 
characterise.  ,  are  expected  between  the  different  types.  The 
results  of  the  ANOVA's  are  given  In  Table  III. 2. 

The  analysis  Indicates  that  the  FPI  overruns  are  more  highly 
correlated  with  inflation  than  are  the  other  types.  The  R  value  is 
.26  which  Is  significant  at  the  952  level.  This  Is  surprising 
because  of  the  nature  of  the  FPI  contract  overrun  procedure.  In 
this  contract  type,  payments  are  not  made  above  the  celling  price 
(which  is  usually  about  1202  of  the  target  price).  Regardless  of 
the  severity  of  the  Inflation  experienced  during  a  contract,  the 
overruns  are  limited  to  approximately  202  of  initial  target  costs. 
One  would  expect  this  to  obscure  the  estimate  of  the  correlation 
coefficient  during  periods  of  high  Inflation  for  FPI  contracts. 

The  proceeding  analysis  Is  based  on  many  simplifying  assumptions 
and  is  admittedly  coarse.  The  ne.  result  indicates  a  statistically 
significant  small  positive  correlation  between  inflation  and  cost 
growth  and  no  further  analysis  is  Indicated  at  this  time.  If  It  is 
found  desirable  to  do  so,  a  lot  of  specialized  data  will  be  required 
to  further  refine  the  analysis. 
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Overrun  Vs.  Inflation 


Source  d.f. 

M.S. 

F. 

R. 

t 

FPL 

Treatment  1 

Error  40 

.148 

.051 

2.88 

.26 

1.704 

Constant  term  *  -4.16 

t.05 

»  1.68 

Linear  Coefficient  •  4.13 

F.1 

«  2.84 

CPIF 

Treatment  1 

Error  33 

.024 

.139 

.170 

.07 

.403 

Constant  term  *  -1.75 

t.l  1 

■  1.31 

Linear  Coefficient  *  1.75 

F.l  ' 

•  2.87 

i 

i 

CPFF 

Treatment  1 

Error  121 

2.64 

0.68 

3.87 

.17 

1.898  ; 

j 

'l 

Constant  term  *  -10.8 

t.05 

-  1.66 

Linear  Coefficient  *  10.8 

F.i 

*  2.75 

TABLE  III. 2 
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CHAPTER  IV 


TECHNOLOGY  LEVEL 

It  is  highly  probable  that  the  degree  of  technical  difficulty 
involved  in  the  contract  work  would  have  an  effect  on  cost  growth, 

A  high  technical  performance  level  implies  greater  uncertainty 
in  the  work,  and  hence  a  greater  chance  for  unsuspected  cost 
increases,  while  a  low  level  of  technology  implies  fewer  unknowns 
and  greater  certainty  In  cost  estimating. 

The  CPE  reports  include  a  measure  of  the  level  of  technology 
involved  In  each  contract.  The  contract  officer  assigns  a  grade 
from  one  of  three  levels,  A  rating  of  (1)  implies  the  highest 
level  of  engineering  sophistication  (frequently  Involving  new  and 
untested  scientific  procedures),  while  grade  (3)  is  used  for  the 
lowest  level  of  sophistication. 

A  one-way  ANOVA  ms  conducted  for  the  53  Amy,  30  Navy  and  114 
Air  Force  contracts  separately  and  entire  sample  of  202.  The  results 
are  given  in  Tables  IV. 1  and  IV. 2. 

The  computed  F-ratio  for  Army  contracts  is  highly  significant 
and  Indicates  a  difference  in  the  percent  cost  growth  between  different 
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ANOVA  On  Technology  Level 

_ 

Source 

d.f.  H.S. 

F 

ARMY  CONTRACTS 


Treatments 

£. 

153,798.— 

12.2244 

error 

55 

12,581.21 

Total 

57 

F.05“ 

3.15 

NAVY  CONTRACTS 

Treatments 

2 

3,491.0 

1.08 

Error 

27 

3,239.8 

Total 

29 

F.i* 

2.52 

AIR  FORCE  CONTRACTS 

Treatments 

2 

36,193.2 

1.72 

Error 

111 

21,004.6 

Total 

113 

F.i* 

2.36 

ALL  CONTRACTS 

Treatments 

2 

16,603.445 

.S48 

Error 

1S9 

17,510.73 

Total 

201 

F.05  * 

3.04 

NOTE:  HS  refers  to  the  mean  square. 


TABLE  IV.l 


technology  levels.  The  average  percent  cost  growth  for  each  cate¬ 
gory  Is  given  In  Table  IV. 2.  Notice  that  the  average  for  the  Amy 
contracts  for  level  (1)  Is  about  7  times  the  average  for  the  two 
lower  levels.  It  Is  noted  that  there  are  only  6  contracts  In 
this  category,  and  the  standard  deviation  Is  quite  high  which  could 
indicate  an  outlier  (one  highly  uncharacteristic  observation).  For 
this  reason,  the  Individual  observations  within  this  group  are 
given  in  Table  IV. 3, 

Notice  that  the  ANOVA  given  In  Table  IV. 2  Indicates  no  difference 
in  cost  qrowth  between  technology  levels  for  the  entire  sample  and 
for  the  Navy  and  Air  Force  contracts  separately.  While  this  report 
is  not  Intended  to  compare  cost  growth  characteristics  between  ser¬ 
vices,  this  last  result  Is  somewhat  surprising  and  suqqests  a  compari¬ 
son  study  In  this  area.  Of  course  this  difference  might  reflect 
merely  a  different  reporting  system  between  services,  or  something 
less  drastic.  Finding  out  the  reason  behind  this  seems  a  worthwhile 
endeavor. 
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Group  Data  for  Technology  Level 

Technology  Level 

1 

2 

3 

ARM*  CONTRACTS 

Sample  Size 

6 

12 

40 

Mean 

286.2 

35.6 

51.7 

Standard  Deviation 

322.0 

60.6 

58.4 

NAVY  CONTRACTS 

J 

Sample  Size 

10 

5 

15 

Mean 

25.8 

31.6 

58.0 

Standard  Deviation 

37.1 

30.7 

71.4 

AIR  FORCE  CONTRACTS 

Sample  Size 

34 

27 

53 

Mean 

54.6 

114.7 

55.9 

Standard  Deviation 

72.4 

233.8 

119.1 

ALL  CONTRACTS 

Sample  Sl2e  50  44  108 

Mean  76.6  83.7  54.7 

Standard  Deviation  143.6  188.8  93.9 


A 
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CHArTER  V 

CONCLUSIONS  AND  FUTURE  WORK 

The  various  effects  which  Influence  the  cost  growth  associated 
with  procurement  contracts  can  be  classified  generally  as  endogenous 
or  exogenous.  Examples  of  endogenous  effects  are  contract  type, 
initial  cost  and  level  of  technology.  Inflation  Is  an  example  of 
an  exogenous  effect.  There  are  some  effects  which  would  be  difficult 
to  classify  exactly;  for  example  contract  duration  possesses  elements 
of  both  types  of  effects.  Phase  I  of  this  research  project  Involved 
endogenous  effects.  Phase  II  (the  work  covered  in  this  report)  involves 
effects  which  are  more  exogenous  In  nature. 

The  recently  collated  research  uncovered  several  interesting 
results,  some  of  which  are  surprising  and  Informative  as  well.  Of 
the  effects  studied,  the  one  with  the  most  pronounced  effect  Is  the 

contract  duration,  that  Is  the  length  of  time  from  the  deflnltlzation 

\ 

to  the  closeout.  This  is  not  unexpected,  since  there  are  so  many 
other  effects  which  are  confounded  with  this  one.  For  example,  both 
inflation  and  total  number  oKmodiflcatlons  Increase  with  the 
contract  duration.  These  last  two  effects  require  separate  treatment. 


therefore,  to  assess  the  Importance  of  the  effect  of  time.  An 
analysis  of  total  number  of  modifications  and  the  number  of  modi¬ 
fications  per  year  should  be  performed  to  see  If  this  effect  is 
significant. 

Inflation  was  studied  separately  in  this  report.  Since  there 
are  no  direct  indicators  of  inflation  in  the  contract  file  data, 
an  indirect  method  of  analysis  was  adopted,  and  It  was  found  that 
overruns  and  changes  in  rate  of  Inflation  are  positively  correlated, 
although  the  correlation  Is  snail.  It  was  further  found  that  differ¬ 
ent  types  of  contracts  exhibit  different  correlation  coefficients 
with  inflation. 

The  effect  of  contract  size  (initial  negotiated  cost)  was  studied 
and  was  found  to  have  a  statistically  significant  but  very  mild  effect 
on  cost  growth.  One  surprising  result  is  that  contract  duration  and 
size  are  not  related.  That  Is,  “larger"  contracts  do  not  necessarily 
require  more  time  to  complete. 

Finally,  the  effect  of  the  level  of  technical  difficulty  of  the 
contract  work  was  studied.  It  was  fcxind  that  the  Army  contracts 
which  involve  the  highest  degree  of  technical  difficulty  also  suffer 
the  most  cost  growth.  This  Is  not  true  for  the  Navy  and  Air  Force 
contracts,  however. 

Phase  III  of  the  study  has  begun  and  the  results  will  be 
ready  for  publication  in  about  November  1971.  The  new  data  collected 
during  the  months  ol  March  and  April  1971,  should  be  available  on 
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tapes  for  analysis  by  late  July  1971.  At  that  time ,  the  analysis 
of  the  contract  edifications  will  begin,  and  will  comprise  the 
major  part  of  the  remainder  of  the  analytic  effort.  Furthermore, 
economic  indicators  for  100  of  the  largest  corporations  involved  in 
procurement  will  be  obtained  for  an  analysis  of  cost  growth  relative 
to  corporate  economic  strength.  It  is  hoped  that  this  analysis  will 
yield  information  about  the  relative  Importance  of  the  "buy-in" 
policy. 

If  there  Is  enouqh  time,  the  effect  of  Inflation  will  be  studied 
in  more  detail.  Its  effect  has  been  detected,  but  the  exact  degree 
of  the  effect  is  still  uncertain. 

Several  other  parameters  will  be  subjected  to  analysis.  The 
authority  to  negotiate,  the  contractor's  original  position  as 
reflected  In  the  Independent  Government  Estimates  and  the  contrac¬ 
tor's  original  bid,  the  effect  of  sole  source  versus  a  competitive 
procurement,  and  the  total  number  and  annual  frequency  of  the 
contract  modifications  will  all  be  analyzed  for  effects  on  cost 
growth. 

The  new  cost  analysis  data  includes  information  from  many  FFP 
contracts.  Since  no  previous  analysis  has  been  conducted  on  this 
type  of  contract,  some  of  the  Phase  I  and  Phase  II  analyses  »111  be 
repeated  on  this  new  data.  It  Is  felt  that  a  comparison  of  cost 
qrowth  characteristics  between  FFP  and  all  other  types  of  contracts 
will  be  most  beneficial. 


APPENDIX  I 


INFLATION  INDICES 

The  Inflation  figures  used  In  this  study  are  obtained  from  the 
Federal  Reserve  Bulletin,  January  1971,  page  AS2,  and  are  qiven  in 
Tatle  1. 

YEAR  WHOLESALE  COMMODITY  SEQUENTIAL  INFLATION  RATIO 


1951  96.7 

1952  94.0  .972 

1953  92.7  .987 

1954  92.9  1.002 

1955  93.2  1.003 

1956  96.2  1.032 

1957  99.0  1.029 

1958  100.4  1.014 

1959  100.6  1.002 

1%0  100.7  1.001 

1961  100.3  .996 

1962  100.6  1.003 

1963  100.3  .997 

1964  100,5  1.002 

1965  102.5  1.020 

1966  105.9  1.033 

1967  106.1  1.002 

1968  108.7  1.025 

1969  113.0  1.040 
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The  base  period  for  the  data  of  Table  1  is  1957-1959.  Column 
three  consists  of  the  ratio  of  the  inflated  dollar  value  for  the 
present  year  to  the  frrmediately  preceding  year.  This  figure  represents 
the  "inflation  index"  for  each  year  and  will  be  denoted  by  1^,  Ig»  ...» 
for  IS  years  (1951  through  1969). 

To  obtain  an  estimate  of  the  conditional  expected  cost  growth 
due  to  inflation  given  the  inflation  indices »  let  the  expected  total 
contract  dollar  amount  which  is  affected  by  inflation  be  C,  and  let 
3] ,  a 2 .  ...»  an  be  the  amounts  expected  to  be  spent  during  the  n  years 
of  the  contractual  period.  Finally,  let  and  nig  represent  the  number 
of  months  Involved  in  the  Initial  and  final  years  of  the  contractual 
period,  and  let  Initial  year  be  represented  by  j.  Then  the  final 
contract  cost,  Cj  is  given  by: 


O.l)  Cj  « 


V^lV^l^y*'*4' 


Ij+lIj+2**,Ij+n-2 


^  jfj  an 


n 

where  2  a.  «  C 
i-1  1 


For  analytic  convenience  and  because  the  a^  are  virtually  impossible 
to  obtain,  it  is  convenient  to  assume  that  the  are  equal,  and  that 

wij  3  mg  3  6. 
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The  assumption  of  equal  is  certainly  not  exact,  but  there 
are  compensating  effects  which  tend  to  make  it  more  nearly  so. 

For  example,  a  large  part  of  the  initial  contract  amount  is 
usually  expended  initially  and  toward  the  end  of  the  contractual 
period.  But,  this  ',s  at  least  partially  compensated  for  by  the 
subsequent  contract  modifications  which  tend  to  distribute  the 
expenditure  of  funds  more  uniformly  over  the  contractual  period. 
Equation  (1.1)  then  reduces  to 

0.2)  C{.  j'V')  ?]  „-T  [f+IJ+t+IJ+!  •  ••• 

*  !j+l  •  !j+2  ...  'jtn-2  ([Ij+n-1+1]  j)  fj 

* 

Cj  is  considered  the  best  feasible  estimate  of  the  expected  contract 
price  at  termination  of  contractual  period  if  there  are  changes  due 
only  to  inflation. 

The  numbers  Ij  are  easily  used  to  calculate  future  estimates  of 
inflation  including  linear  and  higher  order  trends  by  a  simple 
weighted  average  procedure. 

It  might  also  be  pointed  out  that  If  Hj,  Hj+i ,  ...,  Hj+k  are  a 
set  of  coefficients,  derived  from  the  wholesale  commodities  Wi ,  W2, 
....  W19  in  Table  1,  similar  to  the  Ij  but  all  normalized  to  the 
year  (j-l)(instead  of  sequentially  by  proceeding  year),  then 
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wt 


*  2^  «•  •  •  9 


Hi* 


Wj-1 


19, 


and 


T  .  T  •  T  ...  ,  .!M  .  H 

!j+1  j+2  j+k  “  Wj-i  J+k' 


This  equation  (1.2)  reduces  to: 


0-3)  ct’~  „-T 


2-Hj  +HJf1 


Although  the  computational  burden  associated  with  1.3  is  far  less 
than  that  associated  with  1.2,  the  former  requires  a  new  set  of 
coefficients,  Hj,  for  each  possible  contract  initial  year  and  forfeits 
the  advantage  of  the  previously  mentioned  averaging  process  for 
extimating  linear  trends  with  the  Ij. 

The  contract  cost,  C,  in  equation  1.2  can  be  factored  out  of  the 
right  hand  side  and  the  remainder  of  the  tern  may  be  denoted  simply 
by  I.  Then  this  equation  may  be  written, 

(1.4)  Cj  =  C*I 

I  is  the  aggregate  inflation  index  for  the  contractual  period 
used  in  this  analysis. 

In  periods  of  constant  inflation, 


U  '  l-5+i  *  ...  *  I4i  ,  [and  this  is  1  during  periods 
J  J+>  j+n-1  -  inflation! . 


of  no  inflation], 


Cj  then  reduces  to. 
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Therefore,  if  no  inflation  occurs  throughout  the  entire  contractual 
period,  and  the  contractor  uses  the  present  inflation  index  as  an 
estimate  of  the  aggregate  index  I,  then  the  expected  Increase  (or  decrease) 
due  only  to  Inflation  (or  deflation)  is  zero. 

The  analysis  In  Chapter  III  will  consist  in  examining  the  contract 
data  for  correlation  between  overruns  and  the  Individual  contract 
aggregate  inflation  indices.  The  amounts  spent  per  yea  (the  a^),  the 
quantities  m]  and  m2  and  the  amount  of  inflation  planned  for  at  the  time 
of  contract  definitization  are  not  known  to  us.  It  is  therefore 
impossible  to  analyze  In  detail  the  relationship  between  the  corrected 
and  uncorrected  for  inflation  and  the  corrected  amounts  of  money  spent 
for  each  modification  and  for  the  final  closeout  which  is  directly 
attributable  to  inflation.  For  this  reason,  the  coarser  analysis  has 
been  undertaken  in  an  attempt  to  detect  inflation.  If  complete  and 
accurate  data  were  available,  the  exact  amounts  of  the  effect  of  Inflation 
towards  cost  growth  could  be  assessed.  However,  the  coarse  analysis  can 
be  expected  only  to  detect  the  presence  of  inflation.  Mote  that,  if  the 
analysis  does  not  indicate  the  presence  of  an  inflationary  contribution  to 
cost  growth,  then  because  of  the  crudeness  of  the  model  (due  to  the 
assumptions  which  were  caused  by  a  lack  of  data),  it  can  not  be  said 
with  confidence  that  the  effect  does  not  exist. 
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APPENDIX  II 


ELEMENTS  OF  MODEL  BUILDING 

Statistical  analyses  can  be  classified  roughly  into  one  of 
two  categories.  One  category  is  that  which  occurs  when  the 
statistical  analyst  has  a  set  of  data  which  is  generated  by  a 
known  model.  This  model  may,  of  necessity,  contain  many  para¬ 
meters;  but  all  possible  parameters  are  accounted  for.  The 
analysis  is  usually  straightforward  and  inferences  are  drawn 
about  the  relative  importance  of  the  various  parameters. 

The  analysis  in  this  study  is  of  the  second  variety.  The 
data  is  generated  by  an  unknown  process;  that  is*  the  process  my  con¬ 
tain  parameters  which  are  not  known,  or  not  even  suspected  to 
belonq  to  It.  Frequently,  the  analyst  pretends  that  the 

model  is  known  and  proceeds  with  the  standard  statistical  tests, 
overlookinq  the  fact  that  the  model  must  first  be  constructed 
and  then  the  analysis  conducted.  Ideally,  the  statistician  can 
employ  the  methods  of  experimental  design  to  partially  overcome 
this  problem,  but  it  is  rarely  possible  to  design  an  experiment  for 
qatherinq  economic  data.  The  following  is  presented  to  aid  the 
uninitiated  reader  In  interpreting  the  results  of  this  study. 

To  introduce  the  problem,  assume  that  the  data  points  (x^.y-j), 

....  (xR»yn)  have  been  generated  by  an  unknown  model  which  we 
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assume  to  be  *  Sq  +  where  the  are  known  numbers 

and  the  y^  are  the  observed  random  variables  subject  to  the 
individual  error,  z^.  It  is  well  known  that  the  total  sum  of 
squares  can  be  decomposed  as  follows: 


E(yi-y)2  =  s(yi-yf)2  +  s(yi-?)* 

where  j?  =  1-  £yj  and  represents  an  estimate  of  60  +  6-jx-f  or 
E{y-().  In  words  this  can  be  described  as, 

/  Sum  of  squares \  *  f  Sum  of  squares  \  +  /  Sum  of  squares  A 
\  about  the  mean j  I  about  regression  J  (due  to  regression  J 

The  sum  of  squares  about  the  mean  Is  constant  (does  not  depend  on 
which  model  is  used  to  obtain  y^).  Intuitively,  it  seems  clear 
that  reducing  the  sum  of  squares  about  regression  by  using  a  more 
accurate  model  would  increase  the  sum  of  squares  due  to  regression. 
This  would  increase  the  value  of  the  F-ratlo.  However,  the  degrees 
of  freedom  associated  with  a  "larger"  model,  then  has  a  compensating 
effect  on  th*c  ratio.  The  following  remarks  should  make  the  pre¬ 
ceding  slightly  less  qualitative. 

Suppose  that  B*53  (fl0,  Bi ,  ...»  Bp)  is  a  se'  of  parameters  which 
generates  data  according  to  the  linear  statistical  model  y  -  Xb  *  e 
where 
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1  X<|  i  X21  ...  Xpj 
1  X12  *22  XP2 


and 


*  xln  x2n  ••*  xpn 


e ' 


•  •  •  * 


The  elements  of  the  matrix  X  are  known  and  the  are  random 
error  with  E(cf)  =  0,  and  Var  (e^)  *  °  It  Is  well  known 
(Grayblll)  that  the  least  square  estimates  for  the  el  are  obtained 
by  the  equation 


X'X8  =  X-y,  or  B=  (X'XrVjr 

If  X'X  Is  full  rank.  Also,  E(B)  =  e  and  Cov  (B)  *=  o2(X'X)‘]. 
The  repression  due  to  8  Is  8'X'^and  the  usual  F  statistic  is: 


^0 


rx'£ 

(ri -  i'*y) 


p+i 


-v  p 

p+1  ,n-p>l 


Suppose  now,  that  a  sta'  s1cal  analysis  has  been  conducted 
on  data  qenerated  by  the  above  model,  but  that  only  r<o+l  of  the  parameters 
have  been  Included  In  the  analysis.  That  Is,  the  model  used  Is, 
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£  *  *lt!+e  were 


"  ""  ^  fs  correct,  then  the  apparent 

least  square  estimate  of  yj  is: 


It  is  readily  apparent  that  yj  is  not  in  general  unbiased,  fate 
that. 


(4) 

V 

[XljX2]  = 

xrxi  ! 

! 

<VI 
X  J 
% 

X 

. 

X'X  * 

V 

L-'" 

V*a  J 

and  the  "normal"  equation  X'Xs  *  X  £  yields  X-j  X^  =  Xi  \ ,  so 

that  E(£j)  “  Y i  +  (X]>Xi)’1Xt‘X2Y2.  Only  when  Xl'X?  B  0  is  f, 
unbiased  for  rj.  This  cannot  be  assumed  in  the  above  situation. 

The  F-ratio  which  would  be  used  In  this  situation  (incorrectly) 

is: 


(5) 


,  W*  .  (n-r-U  . 

0  ’  (fl  -  t)  '*1  'll 


The  expected  value  for  the  denominator  (mean  square)  is: 


E  ji'Cl-Xi(X1'X1),',X1']jr/(fi-r-l)  - 

aZ  ♦t2'X2'n-Xi(X1-X1r1X1^]  X^/fn-r-l) 

The  numerator  expected  mean  square  Is: 

E(fj'Xi'£)/r*l  *  E^-X^XrX!)"^!  y)/r+l 

*  G2  *  -  +  Ttl  'xl  'X222+T2X2X1T1 

♦  r2'X2'X1  (X,  -X1  )_1Xi  'X^l/r+l 
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r 


'i 

i 

/ 


The  apparent  E.M.S.'s  are  o2  and  Ar/Wj/r"  respectively, 
which  means  that  the  true  critical  points  could  change  drasti¬ 
cally  from  the  apparent  ones.  In  order  to  gain  more  Insight 
into  this  problem,  we  should  compare  the  F-statistlc  used  In 

the  analysis  with  the  F- statistic  derived  from  the  true  model. 
The  true  F  is: 


(6) 


f  -  -Xy  '  .  fca-lL 

it  ■  e'X'y  r+1 


XfXl 

VX2 

x2-x, 

X2'X2 

Compare  this  with  the  apparent  F,  eguatlon  (5).  To  simplify  the 

comparison,  one  Is  tempted  to  decompose  Into  Its  components  X, 
and  X2.  We  note  that, 

r . ,  ^ 

(x'xr1  = 

and  this  may  be  shown  to  be: 

rxTx1-x1x2{x2x2)-Tx2'x1r1  -(xfx,  r’xfXjfXjVxix,  (x,'x, 

-  (X2X2 )  -’x 'X,  [X{X,- X,'X2(X 'x2 }  ->  X  jx,  ]  [x^-x'x^xfx,)-!*^]-! 

NOW  consider  this  from  the  multilinear  model  used  In  the  present 
research.  The  model  is: 


*1  =  B0  +  +  ...  +  sDXpf  +  c. 
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The  "corrected"  model  Is  more  mathematically  tractable  and  Is: 

yi  *  60'  *  ^{Xn-Xi)  +  ...  ♦  Bp{xprxp)  +  £1. 

It  Is  assumed  that  the  analysis  is  conducted  without  knowledqe 
of  all  parameters  so  that  the  analyst  actually  uses  the  following 
model : 

y1  *  60'  +  B1(X11-X1)  +  ...  +  Br(Xr1-Xr)  *  ti 
where  0<r<p+l •  Then , 

a  (B0‘  81»  M  and 

*2  *  (Br+1*  ®p^*  norma^  equations  are: 


X'Xg  *  X'y  and 


where  Suv  *  ^  (Xut-XuHX^-Xy)  and 


The  F-statistic  for  the  hypothesis  6^  =  0  is: 

F°  =  s(yrv)2  -  (Vly  +  ...  +  srsryj 

where  the  are  stained  through  equation  (4)  with  X^  =  X'X1  =  0, 
and  ®  The  F  which  should  be  used  is: 

_  (n-p-1) 

Fq  ~  - ^ - - - - - - - - 

E(yj-y)  -  (^sly  +...+  Brsry)  -  (Bp+jS^  y+...+  80spy) 

Now,  each  of  the  terms  B^Sjy  >  0  since  they  are  of  the  form  r'X'y.  * 

1  X  X  y  >  o.  Thus,  when  testing  a  simple  hypothesis  with  the  Incomplete 
model,  the  denominator  of  the  F-statistic  will  not  Include  the  terms 

-<*r+l  Sr+l,y+’**+  Mpy^*  ^as  e^ect  of  decreasing  the  value 

of  the  test  statistic  Itself,  while  the  coefficient  n-r-1  > n-p-1 ,  and 
this  tends  to  Increase  the  value  of  the  test  statistic.  The  two  effects, 
however,  are  not  completely  compensating  If  n  Is  rather  large,  say  200. 
Even  If  p  =  20  and  r  3  1,  (n-r-1 )/(n-p-l)  *  1,106.  a  rather  small 
increase.  In  this  case,  the  neglected  terms  in  the  denominator  could 
easily  deflate  the  F-statistic  below  the  significance  level. 

The  above  remarks  are  qualitative,  but  the  following  conclusions 
are  worthy  of  note.  The  present  study  Is  of  the  “model  building" 
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variety.  There  are  unquestionably  many  causes  of  cost  qrowth, 
therefore  many  parameters  in  the  model.  Initially,  when  testinq 
a  parameter  for  siqnificance,  a  qreat  deal  of  latitude  should  be 
allowed  in  comparing  the  F-ratios  with  the  critical  values  at  a 
qiven  confidence  level.  As  more  and  more  significant  effects  are 
found,  the  F-ratios  become  more  accurate  as  the  previous  discussion 
indicates.  This  implies  that  as  Important  "contributors"  to  the 
model  are  discovered,  they  are  Included  In  the  model  for  further 
analysis.  At  this  point,  that  capability  has  not  been  included  in 
this  research. 
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